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ABSTHACT 

Phase 1 of the project consisted of a survey of the 
states to deter mine the type of aodel used ip the states' evaluation 
of vocational-technical education. In aadltion, a search was inade of 
EIE and CUE to deterffilne the most recent evaluation research studies 
conducted in each state. Through responses to inquiries and the 
literature search^ the type of #,valuation. and research studies 
conducted in each state was aetermlned. The various ttefehods of 
evaluation used were grouped into the eight categprtes of self^study^ , 
visiting team, follow-^up, employer--basedy coat- tenef it, behavioral 
objective^ nonbehavioral objective, and job placement relatedness. 
Results are presented as discussions in which the literature on each 
of the eight identified evaluation Bethods is suaBarized, A 
discussion of process verst^ product evaluation is also included. 
Tables indicate the t.ype of evaluation used as reported by the states 
and also the number of reported research studies conducted in or for 
the states according to type of evaluation method. Appendixes contain 
project correspondence and a 19-^page bibllpgraphy , (HJ) 
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mvimi OF LITSRATORE 

PROCmURE 



In Novmber, 1973, a letter was sent to the State Departments of Public 
Instruction of the- 50 states requeatinB information on the state evaluation aodBl 
for vocatlofial-teclmlcal education (Appendix A) . A latter and/or naterials were 
received from twenty-two states as a result of that initial mailing. 

After a meetinfs with Dr. Robert Harris at Indiana University In AuRust, 1974 
a letter requesting' Information and tBaterlals was sent to four states he suggested 
as having a system hut who had not responded to the November, 1973 Inquiry. These 
states were California, Georgia, Illinois, and Oklahoma, and a response was re- 
ceived from each of these states. 

In early October, 1974. a "Probe" was made of the Educational Resources Infor- 
mation Center (ERIC) through Indiana University. A Probe ' la a computer retrieval 
program developed at Indiana University to search the ERICTAPES of RIE (research 
reports and studies) and ClJE (loumal articles). The result is a computer print- 
out giving an abstract of the most recent 100 research reports or studies md a 
full citation of earlier relevMt reports or studies. The same Is true for journal 
articles. 

The following descriptors were used for the "Probe j 



1. 



Vocational education ll- Program evaluation 



2. Agricultural education 12. Ivaltiatlon techniques 

3. Business education 13. Educational accountability 

4. Cooperative education 14. Evaluation criteria 

5. Distributive education IS. Course evaluation 

6. Health occupations education 16. Statewide assesantfnt 

7. Prevocatlonal edOcatlon 17. Statewide planning 

8. Technical education ^ 

9. Trade and Industrial education 
10. Occupational home economics 

Anything Including a descriptor from #1-10 in combination with a descriptor from 
#11^17 was printed. A total of 800 research studies and 248 Journal articles was 
Identified through this procedure. The neKt step was to eliminate Inappropriate 
studies from further consideration on the basis of the abstracts provided for the _ 
first 100 listed. Abstracts for the remaliiing studies were located In the Research 
in Education" journals at Wabash College and reviewed at that location, ^proxi- 
mately 350 research studies were actually reviewed on microfiche by two NOT staff 
fl»inbers at Purdue Uhlverslty. In addition, approximately 50 Journal articles were 
exasmned. Infonnatlon was written on 112 r«8eardi,etudi« and twelve journal 
articles. A blblloiraphy is attached. 

During October, 1974, a letter was sent to the Voeatlontl Education Divisions 
of the itate Departments of Idueatlon which requeated updated materials from 
states who had responded the previous year. An jntroducto^ "**-S!|"*'* 
Inf option was sent to States that had not responded to the Noveotoet, 19/ J, 
Inquiry (see Appendix B). 
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In December^ 1974, another letter was sent to the Directors of the Division of 
Vocational Education of the various State Departments of Public iMtructlon who 
did not respond to the October » 1974, letter. Mr. Phillip Mann, Coordinator of 
Evaluation, Indiana Division of Vocaklonal Education, gave us permission to use his 
name as a reference for the project for this mailing (see ^pendlx G). 

To dates April 1, 1975, some response has been received from 45 of the states - 
Thirty-four states sent materials and/or evaluation procedures, elRht sent a letter 
stating their InvolveMnt in vocational education evaluation, and three Indicated 
the state did not have a systeiti. 



II. STATE SYSTEMS 



nirough responses to Inquiries and research studies, the type of evaluation and 
research studies conducted in each state was determined* Table A presents a sumary 
of the evaluation £:Ctlvltles reportedf^ln a state through a response to the various 
inquiries noted above. Since five states did not respond to the letters requesting 
Information^ 'no reply- is indicated for chose states. The various methods of 
evaluation used in the states tend to fall Into eight categories^ 



1. Self-study 

2. Visiting team 

3. Follow-up of students 

4- Employer and /or Community 
evaluation 



5. Cost /benefit analysis 

6. Behavioral objectives 

7. Nonbehavioral objectives 

8. Job placement relatedness^ 



Four other categorlea are Included In Table A; 

1. Undetermitted I incofflplete data 

2, In process of developing system 
3* No system 

4, No reply, 
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* Suhmltted an evaluation of two career education projects In Georgia which were 
developed at the University of Missouri. 

Submitted the annual evaluation report preparad by the State -s Advisory Council 
for Vocational Education . 

. ^ 
Submitted an evaluation system for Career Education Projects, 

**** The system and Instruinentatlon is in the process of being field-tested. 

Table B indicates the number of reported research studies canducted in or for 
a state and other reports specifying a state. Infomatlon collected from the 
various research stud^^s reviewed that could be determined as pertaining to a particu- 
lar state was categorized as follows^ 
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1. fialf-study 

2. Visiting teath 

3. Tdllow--up of students 

4. ^ployer and/or Coimunlty 
evaluat ion 

5* Cost /benefit analysis 



6* Behavioral oblectlves 
7, nonbehavloral objectives 
fi* Job placement relatedneas 
9 . Other 
10 » General disdusslon — ^ evalua- 
tion in generals not specific 
methods. 



TABLH B 

Review of ERICTAPES and Hisc€llaneous Studies and Reports 
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Alabama l 
Alaska 

Arizona i 
Arkansas 
California 7 
Colorado i 
Connecticut 
Delaware ^ 
Florida 
Georgia . 
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idabo 
Illinois 
Indiana 
Iowa 
Kansas 
Kentucky j * 
Louisiana 
iiaine 

toryland 2 1 
Massachusetts 1 
Hichiigan I 
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III. METHODS OF EVM.UATION 

"InfoTOatlon theorists rafar to error In eommunlcatlon as noise* The overall 
literature on evaluation is extremely noisy, has wide gapSs la dlffleult to foliow* 
is voluminous and contradictory ^ and can leiive a reader in a state of frustration, 
(William Stevenson and William Gary Ward, 213) 

An overwhelming variety of combinations of the various methodfe of evaluation was 
found to exist In the available literature. Each metjiod will be>dlp»ussed below under 
the general type of evaluation that It representsi process or product evaluation.^ 
A select number of the various combinationp of methods will then be prasented and 
discussed in more detail. 

Much has been written oi^ the pros and cons of process versus product evaluation. 
Arguments are pLtsfntrnd as to why product evaluation is the better approach, wh^f pro- 
cess assessmeut shbuld be conducted Instead of just looking at -the product., and why 
both types of assessments are essential. 

Many researchers feel the student should be at the crux of any evaluation, Tt 
is argued that there can be the most up-to-date facilities^ most Innovative teaching 
mstfiods, a well-functioning administrative body^ and every other^ f actet ot a program 
can be "'optin^ls ' but if the students cannot find amplo^ent where their training 
will benefit them or where the training can be of direct* use in some manner ^ the pro- 
gram should be modified or dropped from a sqhool-s offerings. This Is basically the 
philosophy of the Individuals who favor ptograra outcome or proauct as the sole measure 
of a vocatlonal-technleal education program -s suecess. The heart of the evaluation 
problem, these people feel, is the congruence between actual outcomis of thp program 
and the objectives of the progrmn. The Missouri Advisory Council on Vocational 
Education (137) favors product evaluation, which S . . would also help to substan- 
tively answer the charge that this evaluation is a small North CTOtral. ' The majority 
of the accrediting commissions throughout the country appear to concentrate on process 
evaluation, , The council felt the evaluation could be eKpedlted If evaluation would 
focus more on output and less on proci^ss, ^ * 

Traditional evaluation has hmoti toward process. Many individuals apparently 
still feel that this is the route that should be followed. Comback In The Method- 
ology of Evaluation (196) by Michael Bcrlven comi^nted^ •Evaluation used to Improve 
the course while It is still fluid contributes more to Improvement of education 
than evaluation used to appraise a product already on the markets" The Puerto Rico 
Advisory touncil on Vocational, and Technical EduGatlon (175) states that evaluation 
should be actually a process rather than a "post mortem'* activity, Th4 authors df 
the Annual Report for EDUTIK, Inc., (12) see both an advantage and disadvantage to 
the process approach, '^Process ewluatlon is a necessary and valuafcle tool by educa* 
t^rs* for educators and with edi'**ators. 

In addition to the people who expound only product or only process evaluation, 
there are many Individuals ^o feel a mixture of process and outcome evaluation 
would provide the best and most useful results. William R. Grlve (87) Indicated, 
^It becoTCs apparent that currlculiffli evaluation Mid developMnt cmnot hm separated 
from the appraisal and evaluation of Individual corapetenelea and that neither can 
be accomplished without an analysis of job requirements and the development of 
criterion measures based on these requlreTOnts. " 

Robert T. Wlllims. (24^*) stims up the process versus produet approachf "Regard- 
less of whether the evaluation Is product-oriented or process-oriettted^ Its value will 
be^ largely determined by ^en md in what form Itg results reach the deelslon-makepr 
and by his coranltment to include Its results in the planning process. I challenge 
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TABLE C 
SELF- STUDY TOTORHATION FLOW 



CoTOiunlty 



Present 
Ob J actives 



(Recant) 
Plana 



1 Stud las 



Faculty 



.StuHents 



i 



— — Staarlng 



CoffliTflttaa 



j Graduates 
! from Special 



Data 



Naw or existing 
conanittaas 



Chalrinan or 
^oord iQator 



New or aiclsting 
committees 



Raaffirm 
approved 
ob'Jeetlvasj 
programs ^ and 
proeedures , 
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Draft 
Self-Study 
' Report 



Discussions ^ Cotmants ^ 
Hearings^ and Action 



Final 
^Self-Study Report 
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Inatitute 
needed 
ehMges in 
objectives » 
pro^rmnii , and 
procedufes 



you to begin tytngi evaluation results directly Imta planning, 
researcher to do It for you." 

^ '* A. fROCESS EVALUATION METOODS 



Do not wait for some 



1. Self-study 

Self-study Is .probably the most commonly used and wrltuen about method c£ 
evaluation In vocatlc nal^technlcal education. Harris W. Reynolds (187) stated, The 
best evaluation Is carried on by the local district as self-evaluation.' The North 
Carolina Department of Public Instfuction (211) indicates that the evaluation/ 
accreditation process of which a thorouRh sfelf-study Is one part, . . . Is telnR 
removed from its historic position of detache_d^lsolatlon to the mainstream of the 
ongoing operational routines of Instructional plannlnR and educational management. 
A stlf-study usually 'is completed By the various components of the school and a 
typical self-study information flow chart by Tadlock Associates, Inc. (98) which 
shows many of the coimnon, components Is included as Table C. Although this partic- 
ular scheme was , developefl for" programs for the disadvantaged and handicapped, a 
'similar format could be i^ed for regular vocatlonal-technlqal education programs 
and fervlces, ^ 

After the Heclslon to conduct a self=study has been made, the school u^ally 
appoints a ateeitng commAtee which will be In charge of the actual evaluatlbn. The 
Individuals directly concerned with the program's (or programs ) conceptual Wat Ion, 
design, operation, and continuous .implementation are the logical persons to be con- 
cerned with the evaluation process and should therefore be on th> steering committee, 
m addition to rractltloners. It is often considered desirable for studenea and 
community reprosentatives to be included on the self-study teams In order to have 
Input from all concerned audiences. These people are Benerally utilised as chairmen 
of sub^^cOBTOittsas . 

^ca a steering eoOToittae Is establishad, the plans for the self-study need to 
be made. The first st^p the coimslttee usually cakes Is to develop a time table _ 
stating what Is. to be done and when. Table D gives eKamplea of time tables used In 
three different evaluation systems* 

TABLE D 
S>i!PLE TIME TABL 



COPES Sehedullng Guidelines 
(self-study and Bite vls^t) 



Step Action 



1 
2 

3 
4 

5 

6 

7 



College request far aaslstatiea on self-appraisal 
Initial planning of COPES su:&8ystem activities 
with chief college administrator 
Appointment of COPES site visit team ^ 
Orientation and plannlat visit to college by 
sits visit chilrman 

Coi^ietian and retu^ of perceptions Instnimetits 
to COPES service center 

Site visit and oral presentation by ^te visit 
chairnan . 
Written summary of oral prasentatlo Ti / 
*Based on flve--day i^rk.week* 



HaKlnum Elapsed. 
Days* FEom K^quest 



5 
40 



20 

35 



55 
.65 



from: 



"COPES Guides Comanmlty ^College OFCupatlonal Programs Evaluation System, 
.California GomBimlty Colleges, Sacrmwnto, Calif ornla. , 
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SUGGESTED TIME TABLE 



Step 



1* Notification of 
avaluatlon 

2. Initial conference with 
adinlnlstratien and 
faculty 

3. Implementation of ^ 
self ^atudy 

4# Sub^cOTiftnittee reports 

5. Steering conanlttee 
final reports 

D*P,I. Review of 
aubmltted material 

7* On--slte evaluation 

8. Haport of evaluation by 
D.P,I. 



Elapsed Time frotn Stare of Evaluationr 
(in TO3nths) 

) i 2 3 4 5 6 7 



^adapted from? ''Evaluative Criteria for^acational Technical Programs by Harrls4^ 
W, Raynolds and others, Pannsylvanla State Department of Publli 
Inst^Msctlon J Harrlsburg* 

' After a time table Is designed erltarJ^ for the evaluation should be developed 
(If not already available) and should examine conditions , pr©^€l©ngs and character- 
istics supposedly found in *'good^ programs^ Most of "the studCes reviewed ha^ criteria 
listed under a variety of headings and sub-headings. TOese .included i 



1* 
2, 
3. 

5. 

6. 
7. 

8. 



Mmlnlstratlon and Supervision 9^ 



Coimntmlty Resources and 
Involvement 

Public ^Ig^lons ^ 
Student Selgctlotk and Testing 
Physical ^Faciilties and Equlpmerit 
Placement and Follow-up 
Vocational-technical Student 
OrganlEatlons 
Advisary Ooamlttee,p - 
Studenc^s^wl^ Special Needs 
OccupatloMJ/ Work fiicpsrlence 

Some aelfi-studles covfetad only a few of thlse areas wlille motm involved o^st^ If not 
allt of the above cate^rles. ' , ^ 



Staff 




Program Planning 


IP- 


Curriculum > 


ll. 


Instructional Content 


12, 


Adult Education i 


13. 


Instructional Hatarlals 


14. 


and Supplies 


* 


Instiuctlonal Actiyitles 


15. 


and Procedures 


16. 




17. 
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After^th^ crlteirla are davaloped, a rating seale neads to ba established for ^ ^ yi 

ffi^^iarffiJAlng. tha^^ Of campatanea ©f ^ach eritarlon meaisure. A variety of rating 

^^iles was found to hm In use^ feut the iwst eoiTOon tended to be a fiva-point scale 
il^s Jollows* ■ - ' /' ' . ^ . . 

5 <©r 1 or VG) " superior * outstandfaig ' ' 

i^' . 4 (or 7/oT G) — exeellant, wall done, ^oya averaga 

% / 3 (or A) r average ■ ^ ■ , ' 

2 ., (or 4 or F) — below average , improvement needed , lacking something 
that is needed 

Y 1 (or 5 or P) poort major Improvement la neaded^ grossly substandard, 

missing ' * 

NA (or 0) — not applicable. > 



After each Individual criterion statement is rated ^ a suiroary is riSually assigned 
to each area or propram* A different type^of rating scale is used In Virginia (91): 

'--^ tiajor improvement needed critical weataiess or Inadequacy exists In 
' meeting the minlmmn standard for th^ guideline* 

I ^ — Improvement needed ~ with mtoor changes the program could be Improved 

to meM the mtoimum standard for the guideline, / 
S — Prog^& meets the minliaum st^dard^^r the guideline, 
f Program eKceeds j^he minimum ^randa^frfoic ttie guideline. 
f Program far exceeds. the mlnimiim stfflidard for the guideline, 
NA ---- nbt applicable* ^ i . ^ / 

While it" appears this rating scale follows the typical 1-5 scale described above ^ 
Virginia State Board of Education designed the ^rating system with sytAolle letters 
<ather thm VG, G* A, P or 1-5) In an attempt to have the over^l program rating 
so that it la not an average of numerical ratings , but considers spedlflc /Strengths 5, 
weataiesseSj and ^^^d^ that are revealed* ■ ^ ^ 

The next step In a self-evaluation generally Is to eenduct the evaluation 
according to the plan established earlier* In addition touting criteria in a 
variety of areaSi questionnalrea and/or tnterviews are often used In an ^ffojpt to 
obtain input from a variety of sources* This utilisation gt quest lonnaliw^and 
Interviews la tSE^t popular ^when the >s©ple looKlng at the program aspcicti are -all 
educators within the scKooi* By seeWlng input from students * business and industry, 
parents, and othef coTOiunlty representatives ^ ,a broader pool of knowledge and diver- 
gent vlewpoirits aid dimensions to the program evaluation* Rather than using 
.questionnaires or Interviewing a sampling of the students and cormunlty peijple, some 
studies Btress comiamlty and student representa^on on the sub-comalttees that rate 
*lndlvl^^l programs sor, depa . - ^ 

A general hearlr.g is sometimes hel^to review and siOTnati^e findings followed 
i by/ a written report^^th commendatloni and^reeonmendstlons. Strategies for imple- 
mentlng the recoinmenditlons to Improye tha progrmns should then be undertaken. • 




, f , Many, stat^is utilize and a number o£ Research studies focus on the use of 
^ visiting teams as an eviluation method (Tables A and B). The specif Ic Mthods for 
.J. thi team .revle^^/4re widtly varied. 

Y Ganerally, the flrit step In the visiting team evaluation is to select a chair* 

" man- There did not appw to be my patte™ dealing with the selection of a chair- 
I^^man- One ©% the next steps in a visit Img team evaluation la for the dialnron to 
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visit the school to hm evaluated and dlseuss the entlra procedure with the chief 
administrator and anyone else the adm desires to Include, After such an 

Initial meeting 9 the reznalnder ,o£ the team Is usually selected* This la sometimes 
done with the school administrator Indicating his approval of each team member. 

The nomposttion of the visiting team differs eonsiderably from one study or 
state response to another. Team members are sually chosen from the following llstf 

1. State Departt^nt supeivlsorB for each vocational- technical area 

2* Other Division Vocatlonal^technlcai Education personnel 

3» Personnel from other Divisions of the State Department of Public 

lastruotion c^-^-^ 

4. Vocatlanal^ technical teachers and administrators from other local 
schools 

5. Vocational-technical teaahers and adinlnlstratbrs within the school 

6. Vdcational-^ technical teacher educators 

7* State Advisory Council members ' j 

8. Local Advisory CoTOnittee members 

9^ Business and Industry representatives 

- ,^0* Lay public , - 

11* Students ~ former and present 

12. Other vocational^technical people as needed. 

!^lle some studies include people from the State Department of Public Instruction , 
local sehoplSj and the conntiunitys oth^r studiee use teams consisting solely of State 
Departmerit personnel. Out-of-state people as well as in-state individuals are some- 
times included on teams. 

Some visiting teams^ such as Mississippi -a (228) j, are composed entirely of In^ 
state educators « This apprpach has the advantapie that all team members are familiar 
with strategies arid state policy concerning vocational education. Team members 
als^ benefit from their observations of various programs by noting ways to improve 
their home programs. , ^ ' - 

In other states, public input Icito tlie evaluatibn system is considered eaaential. 
A study done by H, M. Hamlin (95), stressed the Importrace of citizen evaluation of 
public occupational education rather than educators * . R. C. Stape notes (95) , "'It 
is a great misfortune that th£ best trained evaluators have been looking at education 
with a microscope rather than aj^panoramlc vlaw finder-' " ^ 

The school often pre^res for the team visit by cof^Tetlng certain forms and 
questionnaires and gathers materials for, the team mefltoers to ^vi^, T<nien the 
team arrives, there is usually an orientation meeting for team^menibet©^ team chairmani 
and chief administrator of the school. After the orientation maetlng, the various 
team members have their specific assignments" and they start evaluating various ' 
prbgrama and school coi^bnents through pbservation,. talking with various staff 
members and students, an* reviewing TOteflals provided by the school* Soma studies ^ 
also have the team members Intervl^ various comnRinity reprtsentatives* 



'*i^fflr*E^iW'fi'ff^b*f*TOefr*we 

members who assign ratings to the various program aspects. Areas examined and • 
rating sSales used fte alMlar to those discussed unde^ the self-atudy. ftie (or more) 
member is usually assl^ed particular areas to evaluate. After each team metBber 
completes a section, he/she generally provides an overall rating for 'that assigned 
aection and writtes coiiittendatlons and reqoimendatlons . Whan all team fibers have 
Completed their tasks ^ overall cofmendatlons and recommendations are OTltten. A 
kchool fflbetlng Is usually held bef era the team leaves sp the team oeabeps can provide 
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.ah IfflsWdlate feedback of their findings to either the entire atmff or to select admin- 
istrstors and/or faculty Mmbers. A final written report is subsequently subiRltted. 
The finat sufflmai^ report Is usually prepared by the team chalrm^, 

^ - The time allowed for the visiting tmwm evaluation varlee with how eKtenslve an 
evaluation is being dona and how much preparation Is done by the local school. 
Table D und« the self-study shows some time tables that Include vloiting teams. The 
length of tfie a team is at a school varies with the size of the school and the areas 
being evaluated* the duration for a team site visit Is generally from one day to a 
week (it also depends on thfi number of people on the team*) 

The team slte-vlslt method of evaluation is costly since the team members gener- 
ally have to have expenses and honoraium pald« If a local school must pay all such 
expenseSj^ the total figure could ba quite prohibitive* ^ ^ 

^ . B. PRODUCT IVM.UAT10N IffiTHODB . ^ 

1. Follow-up of students 

Ot the various types of product or outcome evaluation , follow-up of students 
Is the moat common method used among the states. The type of informatian generally 
aouaht In such d follow-up is sumarlzed In the objectives as found in th e Final 
Report for the Central Kentucky Vocational Education Evaluation Project (VEEP) tJ5). 



5 They are 



1. To acquire data reflecting the number of former students of the Central 
^ Kentucky Vdcational Education system who (at time of this stud>) are 
^ employed and unemployed. i 
2/ To acquire data on the number of fortter students tn the Central Kentucky 
Vocational Education rfgion who have^ moved from the area in which 
they received vocational training. 

3. To determine wUether or not graduates of the Central Kantueky Vocational 
Edueatlon system are contlnuljag their education* 

4. To determine the reasons foCTwr students continue or do not continue 
their education, £jm \ 

5. To acquire data concerning the time lapse between graduation (exit from) 
vocational education and acquisition of full-time employment* 

6. To acquire data reflecttog the Success o£ the Central Kentucky. Vocational 
Education system Ih educating former students in - " 

a* occupation of their choice i 
b. occupation for which they are qualified 
' level of attainment they desire. 

7. To acquire data concerning the relation of vocational education programs 
taken to the actual occupation of the former students* 

8* To obtain feedback information from fom^r students Including! 

a. satlsfactlonMlssatlsf action with vocational education taken, 

b. recom^ndations for improving the vocatlohal education system, 

c* type of education the graduate wuld have taken In retrospect, and 
d, type of education former students are taking or would take (given 
the opportunity) . 

I < II nufc BP' i iii^ i fcr iii ^ p ^ i i iipniM i f i i i' n I II jti ■ ihi ijiiti 



tloyd L. McKinney and Carles Oglesby (133) cited five groups of people who 
ought to be involved in conducting a follow-up study to insure maximum return of 
valid/ data. They are _ ^ 



1, coimseilng service, 

2* teacher, ^ ^ 

3, school admtolstratlon, lb 

4. student committee, and 
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5, Beard of Eduoatlom and Vdcetlonal Educaclon Citizens' Mvlsaty Conmlttee. 



There ara three baglc ways ©f eonductlng a follow^yp study 

1* Mail questionnaire « poor return and not representative ©f all fonnar 

studeiita^ but the least SKpensive' 
2, In^person Interview eostly and time consuming » but clarifying ques^ 

tlons can b% asked r and 
3* Telephone Interview less costly and time consum^g than to^parson 

Interview^ but limited in the amount of information that can be obtained. 

When determining the procedure to follow ^ McKlnnay and Oglasby feal the following 
savan factors should be ^onsldered^i 

1» t^at kind of data are needed 7 

2. How much data are to be obtained? - . 

3* What are tha sources of data? 

4. How much time ±m available? 

5* What i# the size of the group(s) to be studied? 

6^ How much money is availabla? 

7. VThat is the availability and compatancy of the staff? 

After the type of follow-up Is datermlnad^ the students are r selected. Some 
st.Adies include Just graduates while others also Include dropouts^ McKinnay and 
Oglasby feel It is important to Include the dropout and make him feel that ha la a 
vital part of the, follow-up population. They suggest the ppssibilitles using a 
different approach for the dropbut such as a different form of letter and quastlon-^ 
nalre or avan a different type of survey like an interview* 

A few studlas survay students vlthlin a few months after leaving the school and 
never again. Some studies conduct t follow-up shortly after graduation and ag^n In 
about one year « Thi length of tlma elapsed af ter ^Aeaving tha school aQd conduction 
.of a follow-up tends to have &n adverse effect on^'^the rate of return. For example^ 
the rate of return on a f lye^yaar follow-up will probably be . lower t^an the rata of 
return for a one-year follow-up study. % Therefore^ the daslred rate of retum*5should 
be considared when deciding to conduct a dnm^f two-^ or more year follow-up. 7 
HcKlnney and Oglesby (133) indicated that only students out of school for opa year or 
mare should be included; Students can ba Involved and alerted prior to graduation by^ 
having taacliers discuss ^he import wee of follow-up studies and by having the students 
supply basic information including the naipe and address of soTOone (such as a relative 
In the area) who would know whara to reach them should they nova. 

A maj.1 questionnaire is. the most cosmonly used mathod. Various mailing procep\ 
dtj^res were used In an effort to obtain a good raturnr 'McKlnney and Oglesby (133) 
suggested the following pattern i , ^ 

■ ' - . ' ; - ^- ' . ■ \ . ^ ^ v ■ ; . 

. 1st mailing — '-alert-' card 
2nd mailing ^- follow-up instrumntp cover lettjer» and. return envelope 

(stamped and addrMsed) % ^' 
3rd mailtep ^"_^f t lgy. t?^ok y pu^rji^ ndey card ^^^^^ ^ 

4th mailing sacond request follow-up InstruTO 

and return Envelope (stampad and addressed) 
5t^ mailing second thank ybu-reminder card. ■ 



John k. GoK (52) utlllzad the followtog raalling pattern.* 

ist roalllng — qusstlontiaire and pre-fltaiBp^d, pr#-add*ees«d Mturn 

I enveiop# ' " 

2nd mailing (10 days later) reminder letter to non-respondTOta* 
3rd mailing (10 days later) ~ questionnaire and envelope, 

Donald V* Brwn (30) listed the fqllowing techniqujis to produce a high return 

rate at the laost eoononical costi ^ . 

* 

1. involvement of present students to address envelopes * help locate 

students » and tally responses: 
2^ Wiling a ^'get prepared" card* 
3* a brief yet personal coiBmltTnent cover letter? 

4, a wfll-designed Instrument that requires a mininmm of ttae and effort 
to answer! 
r 5. follow-up reminder cardt and 

6* finally send out another survey and a personal ^ typed letter to non- 
returns* ' * ' 

He also ^suggests that the card /and survey-cover letter be mailed out so as to be re- 
ceived about the same day of the ^eek (i.e., ^tonday) * Tliird and fourth mailings 
should be scheduled to arrive at a later day of the week (i.e., Friday) which compen- 
sates for tht fact that some people are early weeW: while others are late week 
respondents t^ mailings. 

David J* hicel (174) reported a research study finding on how to improve the 
returns on follcM-ups. The study concluded thaL returns could be improved by 
aprraximately 21% If a pre-letter was used ^ . the questionnaire was printed on green 
paper, and a novelty such as a packet of instant coffee was Included with the 
materials, to addition, ftaterials in the packet should be personalised by having 
addresses on the envelope and letter typed and the letter hand-signed, 

Mc^nney and 0?lesby (133) feel a cover letter should be used and should be 
brief* Five items " ihould be Included: ' , 



i 



purposi o£ study 5 
^, uses tc be nade of the flndinRS, , ' 
S. Importaftee of hearlni from everyonS In the class , 
U. a Buggeited/ date by which the form ahould be returned, and 
b. aasuranee of the eonfidentlallty of the Infomaelon to be provided. 



To sthose graduatei who did not respond to the second fflalllngeOf the |ollow-Up 
of 1965 'Gr aduates of Waeonaln Schools of Vocational , Technical. ^ MiOt Education , 
likneih/j. Little md Jlchard W. T^hlnfleld (123) sent out two, additional mailings 
of a' slv>rter form requesting only basic data. Thim procedure Increased ^he total 
return by approxlnateiy 35%. 

Results of follow-up studies are used In a variety of ways such as: . ' ^ 



"^r'^o'l(elefmX«S''3WTlt 

2. To asseis cost /effectiveness or cost/benefit, 

3. To prbvlde program Imptovement sufigeatlons, and 

4 . To prepare State reports . 
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2. iBflployer-'based Eyaluation 



Donald V. Brown <30) stated, "Thm prlBiary function of vbcatlenal-'technlcal adula- 
tion la to provlds quality Mpleyeea to businefls and Induat^- In business and 
industry, the proeess quality is largely a responsibility of the personnel mployed 
(the produet of veeatlonal*techrilcal education). It is generally redognlied that In 
buslneast manufacturing or services the product relates closely to the quality of 
the process. Upon this Idea the Local Evaluation Project attempts Co relate the 
process of education to quality of Its product (the student)," Otlb of the o^thods 
used by the Tennessee Local Evaluation Frojeet is an employer'^based evaluation, 

■ ^ ' ^ 

An tmployer evaluation la usually based on either a mall questionnaire, an Inter- 
vleWs or a combination of the two* A typical evaluation utilising a questionnaire 
was carried out In New Mexico (131). A covering letter from the Governor was sent 
along with the questionnaire. In an effort to achieve a quick return* a deadline 
was established and included In the letter i '^Questionnaires returned after the 16th 
of December may be to© late to be used." The employet rated five statements as true, 
false 4 or no effect concerning program graduates. The statements eKanined ^ere^ 

1. Those who have had this type of training generally need less on^-the-job 
trdUnlng because of Job kriowledge and skills they already pqasess when 

, hired. 

2. Those who h^ve had this type of training lenerally can be trained faster 
or more easily when required' to take , the samd instruction as others. 

3. Tho^e who have had this type of training are potentially more productive 
and /or promotable . 

4. Those who have had tlbis type of training are vrntm likely to posjsess 
specific critical Job knowledge and skills ,r.hat you find difficult to 
obtain or costly to -develop In yeur to^loyaes. ^ ^ 

5. Those who have had this type of training generally can be counted upon ^ 
to have better work habits » attitudes and mbtlvatton . , 

Each of these statements was answered for five types of programs: high school, post- 
high school, adult eduction ^ Federal programs for those with special needs, and 
military* Imployer opinions were also sought concerning programyavailablllty In 
three respects r " . ' 

1. Is thM^ sn pversupply of applleants? . - 
Z.^Is there a need for more applicants? ^ ^ - 

1 3^ Do you prefer to start fresh and train on-'the-^job? 

L^^n H. Wlllett and William E. Plland (243) used both a questionnaire and m 
Interview for emnloyer Input. The same questionnaire was used for all program areas 
because the staff of Moraine Valley CoTOunlty^^College felt^'that a given type of work 
or work situation Involves generally the s^e "job requirements" regardless of the 
activity or situation. The staff from/each of the college's program areas assisted 
In developing the questionnaire items luid^ opan^endelS quest lmis %hich were rtlevant 
to* their area of concern, to Interview schedule was developed to enrich data analysli 
and to gather data specific to individual programs not covered In the quest lannaire. 

A major problem whra carrying out an employer-based evaluation appears to be 
gg-tttfi ng ^, af^f^q^ f g^^f r#sp9pse ^nd. ysefal ^inf opwtlon from the etg|>loyers . If a ^ues-- 
tlonnalre is sfent to a large cpmpanyi someone In per sonneT^MgWi ' easily" "fill It^WT" 
without knowing the studmt or how the student ii* actually doing* The supewlsor's 
time Is ilmlted, and he might not be able to be Interw^lewed. In order to conduct an 
effective employ er^based evaluation, excellent sclwil*-busineas*lndus try rapport 
should already exist If a fflsKlTOm of suceesii Is to hm expected, 

■ / ■ " ■ ' ■ " 
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3*. Behavioral Objeetivss 



Robert T, Williams (244) Indleated that evaluation was the aomparisan be- 
^ean the prooassss, outputs » end outeomas which ara desired and tha processaSp out- 
puts * and outcoMa whloh are actually obsarved. tn this caae, output refers to what 
the system pro duc-ed^ axiA outcome takes Into consideration how this produet was 
absorbed int^ the environment. Ha said there are five levels of evaluation of In^ 
atructional programs in pceupatlonal education' 



System 
Analysis 



Outcome 



Product Performance 



3 



Output 



Input 



The third levels product performancas can be measured with^ the help of behavioral, 
objectives, Mr. Williaros feels this challenges educators to move away from the 
traditional practice of holding time constant and allowing achlaveMint to vary to ' 
holding achieyement constant while allowing time to vary thus provldihg for individ- 
ual differences* this concept of allowing time Instead of achievement to change is 
fairly new, and little has ieen written on the subject. 



Behavioral objectives examine student «nd/or program out cotcs with two measures: 
quantity arid quality. The number achieving spMific objectives as well as the > 
quality of the perf ooawce are used for thjs type of evjluatlon. ^ , 

Th^ Massachusetts Information Feedback System (111) utilises behavioral :obJec- 
tives which have three basic components i 

1. Tell what it is that one will have to do when he. is evaluated^ 

2, The conditions under which he will have to perform, and 
3^ The level or quality of performance expected* 

In order to determine the number of students completing each igs^bjectlvep a pre-test/ 
post--test format is used* Criterion testing is done prior to the beginning of 
tostructlon in a. curriculum imlti and post-testing la administered after the unit, 
has been presented. M analysis and reporting 6f the test results is than provided* 

In the Evaluation Service Center" for Occupational Education * Final Raport (69) , 
it was stated that the perfomimce test used tp maasure the objectives cm be con- 
ceived of as a criterion-referenced test for three reasons? 



2* It seeks to ascertain a subject's possession of a specific co^etency 
ratherlthOT to complete a comparison of the subject -s cbmpetency level 
to a previously measured nom group, mA = 

3. "It usually raquires/^a^dlchotoi^^ as to whether the competency 

has been demonstrated, _ 
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The report "also cited three reasons why perfo^ance testijig is espeqlally approprifte 
for evaluaclng vocaclonai^technioal education i 

1* Performance tests can be hypothesized to produce more relevant and valid 
data concerning the instruc;tlonal program output, 

2. They produce product reco^rds which can be i:.tudled by teachers to dlag- 
nose where a weakness may have occurred , and 

3, It contains the flexibility demanded by the Information needs of an ^ 
evaluation system* 

The Fresno County Department of Education, Fresno, California, has an Autemfited 
system^ PEAPOL (Projgram Evaluation at the Performance Objective Level) (161). The 
system generates a series of reports designed to allow vocational teachers ^nd 
district admlnlstratdrs to closely n»nltor the progress being made and expenses being 
Incurred In individual classrooms. Its approach to the use of behavioral objectives 
is dissimilar to the method described above for Massiichusetts. For each course § a 
set of measurable behavioral objectives (not to exceed 75 objectives) which completely 
describe course content must be developed for the PEAPOL system. A time clock Is 
installed in each classroom and students are provided time cards for recording. the 
amount of time they spend on each objective to the nearest hundreth of an hour* 
Another form used to supply necessary data Is the class event. Input foms. The f 
system generates four weekly reports f ' ^ 



1. The student suranary report shows how madh time each student has spent 
on each bbjectlvaj which objectives he has completed ^ and how his 
present rate or progtess compares to that of ff^evlous weeks. 

2. The classroom suaroary report produces grouped data^lch shfows the total 
f amount of time all of the students In a class have spjint working on 

each objective and how the class's present -'rate of progress compares to 
that of previous weeks. ^ 

3. The dollar summary report displays overall class budget, dollars expended 
to date, prorated coat for each objective , and prprated cost per student 
for^ each objective. , . ' 

^ 4. The special student-report groups students by seK^ ethnic group, ages 
reading or math scoresi, and produces performance data for each group. 

Tfie success or fallura of these two approaches ustog behavioral objectives Is 
not,^ given In the reports rtvlewed.w No Inforratlon was su'^plie^ as to whether these 
systems are currently being used in any schools or states or if they are stilly In the 
developmental stages. ^ " . 

4, Nonbehavloral Objectives 

While .behavioral objectives look at both quantity md quality^, nonbehavioral 
objectives eKamlne only the quantity or number achieving an objective. For exudple^ 
''15 out of 20 students completed the sssll. engine repair progrra," referii to a ndn^ 
behavioral objective because no mention: is mde of the quality of the program output„_ 
(what do each of the 15 students kno^?).= 

/ , . - : ■ y ' ^ ' 1. ' 

^approach to the use of noitfiehavloral^bj act Ives for evaluating vocational* 
technical educatipn was developed at; The Center for Vocational jmd Technical Educa^ 
ti^n^ The Ohio State University, entitled, A System for^Sgatwide Evaluation of 
^ #oeert oftgl'^^* #ducattoff' ^40^"^^^ „ ■ ■ r. , ■ , 

' man agemt!nt"*by*object Ives approach to progrM planning "by voea^tonal education agen-- 
cles. It is also constructed to serve as a monitoring mechanism for providing Inf or- ^ 
mat Ion about the eKtent to which the vocational agMcy Is achieving Its objectives. 
thm authors of the system feel the nmln point in the evaluation-progrra planning cycle 
Is the systematic manner to which the vocational agency proceeds. ^ ^ 
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The mymtrnm has five system elements which prcvlda the data required for 
eyaluatlve decisions about programatlc outeonies^ 

i. A aet of program objeetivas atid goal stateiaents. There are four broad 
objectives and 47 goal statements. Each goal stateinent is designed to 
have a nund^er supplied by the vocational aget&cy in order to quantify 
the goaly 

2* A data set. This was deslgned^tp produce a core of evaluative Infarma-" 
I tlon useful to TOSt states md Is organised to provide? 
a. student In^school data^ 

b« vocational progran data, ' 

school data .m|id 
d# follow-up Inforoatlon about f error vocational students* 
3, A set of sIk Instruments used to collect evaluative Information* 
4» Information flow procedures. Valuation personnel plan for distributing^ 
collectings processing, and reporting evaluation data^ for receiving 
feedback required to improve the system^* and for revising the system In 
preparation for system ra,pycllhg* 
5. Data processing routlties and computer programs to produce data In a 
variety of foraats useful to system users. 



nie objectives for 



Alas^ (125) examines how well program objectives were meti 
each vocational program are to have four characteristics: ^ 

• ^ I ^ ^ ^ . ' ■ 

Im Relate to the accpmpllshment of priorities, , . 

i. Contain a time frame » 

3. Be measurable, and • ' 

4. Indicate degree or time of achievement. 

The results of the various programs are determined by means of a follow-up question-* 
nalre, 

rS/ Cost/lffectlveness^ Cost/Benefit, and Cost/Ef f iciency^ 

T^Thlle there is a good deal written on the subject ^ most of it is eKtremely 
compleK# Although many formulas cited in the studies are ye^ lengthy and appear 
to be compleKy the real compleKlty lies in determining what numbers to "plug" Into 
the formula. The terms cost /affect ivenessp cost/benefit , and cost /efficiency are 
often used interchangably. William W. Stevenson and William Gary Ward (213) .indicate 
the^e are no concrete definitions to those three terms, but they deal somewhat with 
the following I ' 

1, Cost/ef factiveness ^-^ ratio of products to total cost of program. 
- 2* Cost /benefit payback to socisty for its investment in aducation. 
3. Cost/efficiency ration of resources to training activities. ^ 

In tfiiW report, the termsx^ll be used interchangably. - 

Jacob J. Kaufmra (114)^m^ed sIk mlqcpiiceptlons of cost /benefit analysis. 
They are \ 



1* It is merely a subterfuge for seeding to conduct education on gX^least* 



2 . A benef it Is measured only InMollar terms , 

3. Chere are some things that are qumtifiable. 

4. Not fully develolped. 




5. The cost /benefit analyst substitutes hla iudgm'^nt iew that of the 
d e o is ion-^maker a 

6. Tends to l^are political ean8lderation&/ 

' Mr. I^ufman stated that one aspeq^t afi aostVbenef It walysls which ahould be 
stressed is that It is baslaally a *Vay of thinking.*' He indicated it tends first 
to force an adsainistrator to think thrau^h his abject Ives, to concentrate an costs 
as well as objectives , and to think In tenns of alternative ways of achieving the 
same objective^ Qiarles 0* Hopkins (101) remarked, '^^ere Is a certain quality of 
product produced by vocational education that business. Industry, and government 
services wish to acquire « It costs a spaclflc moount to produce the praduct to the 
specifications required # A cost can be aasoclatsd with the accOTpllahmnts of a 
vocational traii^lng program. If the objectives of the schools warrant change p the 
admlnlsttatlan should evaluate the total vocational progrTO by looking at the 
a&Mevements and costs of each individual program.'^ 

Teh-Wet-Hu (102) cited three basic steps In determining the cast/bmeflt 
analysis of a program: . 

1, Identlflcatlan af costs and benefits of a given program. 

2, iUists af^beneflts and costs, both private md social, are expressed 
is monetary values in order to arrive at an estltQat^. of current net 
beheflts. ^ - 

3, Comparison must l)e made of the stream of annual net benefits and the 
— ^ cost stream of the program. 

A variety of factors must be taken Into account in order to analyze vacatlanal* 
technical education as an investment In the human agent. Thtr relationship between 
costs and benefits should be determined only ^en factors such as_-.time, depreciation 
risk, an^ uncertainty, are taken Into account « Below is a description of the social 
and private costs and .benefits as presented by Richard H.^ P . Kra ft (120). 



Social 



Costs 

Def • • costs incurred either directly 
or Indirectly by society at 
large for operation of f oraal 
vocatlonalTtechnical- training. 

Elements ^ 

1. Total dollar amounts actually 
expended for providing educational 
programs. 

Elements 

2. Operational. costs of not leasing 
or renting facilities. 

Benefits 

fief p ! economic welfare gained by 

society at large as a result 
of formal training., 



EletBent^ 

Locals State, Federal, marginal 
taxes attributable to foraal training. 



. Private 

- 'Costs 

Def . I casts incurred either directly 
or Indl-rectly by students for 
forii^l vocatlonai-^technical 
training. 

Elements . 

1. Foregone earnings or operational 
costs of students due to attend^g 
tralJilJig progrus. 

Blsments <i 

2. Total direct costs to students for 
registration p books, supplies, etc. 

Benefits * 

Def. i economic welfare gained by Individ^ 
ual as a result of formal training. 



EletBente ' 

Student *s marginal earnings attrlbutabla 
. to formal training. 



btiiirles 0, Hopkins (IQl) cited an #xM^e of another means tot detarmialng the 
cost /benefit ratio of a pieogram, A Carpenti^ program has 30 scudents anrollad. The 
^prograra length Is two years, 80% of the students cosplatlng the progr™ had a 
narketable sklllV Thm total annual ac^ta of the program ate $ 1 1 ^ 843 , 00 For 30 
students p the cost Is $394,77 per student anhually« School information Indicates 
that 15 students completed the prpgraifij and 25% ©£ the graduates were employed In 
the ar^a for which they were trained or In a related area (four studants) , The 
annual placement cost of these four students Is then $2*960,75, A total of ten 
students are placed In related and nonrelated employ^nt for an annual placfement cost 
of $lsl84.30 to the program, ^ 

Steven L. Barsby (22) presented "The TOree Methods of Comparing Benefits and 
Costs,*" The type of comparison, method of calculation ^ md decision rule are 
presented In Table Ei 



TABLE E 

'The Application of Cost--Beneflt toalysls In Manpower Area,'* 
Barsby, Arizona Occupational Research Coordinating Unlt^ 

The Three Methods of Comparing Benefits and Costs 



by Steven L, 
Phoenix, 



of 

Comparison 



Percent Value of 
Net .Benefits 



Method of .._ 
Calculation 



^0 (1+1)^ 



Decision Rule 



Select the project with 
the highest net beneilts 
first, then pursue 
successive projects In 
descending order of net 
benefits , 



2* Hate of Return 



1 • »t-s 

^ ^ ^0 

t^O (1+r)^ 



Select the projetft with 
the highest rate of 
return (r) ^ then pursue 
successive projects in 
descending order of (r) ^ 
until (r) equals some 
predetermined Interest 
rate (l)ir 



3, B enef It-Cost 
Rati© 



i 

t^a 



n 

t^O 



(1+1) t 



Select the project with 
the highest B/C^ then 
pursue projects In 
descending order until 
B/C * 1 or budget 
e^austed. 



B. •benefits in year t 



C^ ^ costs In year t 
'^•"TT^'^^nraAer^f^'^sM^spafia^ 

4 

1 ^ social dlseoune tsLtm 
r • rate of retuim 
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Very little inferfnation was found dealing with job placemetit relatednass as 
it relates to the evaluiJitloTi of voeational^teehnical programs* In sone eases ^ 
^ this was tied in with a follaw-^up of program graduates, but methods fdr assessing 
the job relatedneas were not provided « 

David N. Wheeler (24^) diaauised three methods., of measuring job relatedness. 
(1) Ask former student if his present Job is related to his training;^ This obtains 
^ only a yes or no response, and the reliability of such responses is questionable, 
Relatedness may mean different things to different people* nils could be mlnliniiedp 
however^ if a deseription-with examples is used to give the respondents a referenee 
upon which to base their answers. (2) individuals rate the relatedness of their 
training progrra to their jobs on several educationally-relevant variables. The 
difference between raters and that the relative empha8j.8 of variables will shift 
between oecupatlonii are problems with this n^thod* (3) An Individual will supply 
his job title along with a brief description of his job. On the basis of this 
information and a Imowledge of the occupational training program^ the evaluator 
would ifiake a judgment aa to the relatedneis of the occupation to training* A sult^ 
able decision model is needed* Mr. Wheeler's paper describes a complex decision 

model for this method* 

■ ■ '■ 

David J* Pucel (174) discussed the design in Minnesota to provide information 
on job placement relatedness and program improvement « Mr* Pucel Indicated that the 
overfill ef fectivenesa of programs should be judged on the eKtent to which the grad-- 
uates of the program become employed and maintalti employment in occupations in 
which they can utilise the skills developed in the program* Five classifications 
are used for a graduate's job one year after graduations 

1* Employed in exact occupation -for which trained* 

2^ Employed in occupation related to but not. exact occupation for which 
trained V 

3, Jtaployed In unrelated occupation i 

4. - Unemploy ad seeking work but cannot find work.' and 

5* Unavailable for employment further trainings military service, 
/ illness s housewife^ or pregnancy* 

, COMBiNATION OF ftETOODS^ 

in addition to each qf the methods discussed above being used for evaluation. 

. a variety of combinations of these methods was found among the lltarature* ihe 
most widely found combination is a self-study followed by a visiting'^ team* Often 

" the visiting team's evaluation is extefisivep but sometimes it is mainly to verify 
reportings on the self-study. Follow-up of graduates md nongraduates frequently is 
also used with the self^study and visiting team approaches* o ^ 

toother conbinatlon that is found is an employer-abased evaluation and a follow- 
up of students* loth of these methods are also used for cost /benefit analysis md 
the behavioral and. nonbehavioral objectives approaches* 

' IV. CONCLUSION / X 

I^ile there is a greats amount of literature on vocatlonal^teehnlcal education 
^ evaluation J each approach has several advantages and disadvantages* There tm no 
M^ioi^to»o&-.matho4^£^Mluiy^ rf or eye^^^^ to ^^^^ 

proRram ^Ich would always provide the most useful information. ItotM^lBviTop^g' 
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an evaluaVlon system, aeverml factors should he. corfslderedi 



1. Is the evaluation for the entire institution or Just certain areas? ' 

2. Db' you want- io avaluate vocational-technical programs 'or co^^ra&s? 

3. What Is the purpose of the evaluation? ! ' ' / * '* 

4. How much time flo you have to conduct An evaluation? . . ^ 
5* How much money Is available? ^ , v " « . 

6* Are there any state or federal requirements that musf be fulfilled?' 

7. Whats is the targe^^ f or the evaluation (i,e*, students, ' faculties , 
V staff)? \r ^ , " , ; ^ 

8. ,,Do you want to evaluate the procegse'Sj the products^i.-or both?^ 

9. How B^ch cooperation can be anticipated from the school staff. 
Advisory Conmiltteess and the comiunlty? . / 

10. Are resource people available to ^assist when needed?^ 

11. Should s^eaial groups receive addltiional attention (i»e, disadvantaged , - 
handicapped y etc.)? ^ ■ ^ - ^ ' 

No matter what design Is used for evaluating a particular program or 3chopl, thfe 
most important criteria to keep In mind are that the system must meet the n^eds 
of the school or program, and a desire ahd cooperation to ImpleAent th'e results 
to improve the programs should be evident on the part of all concerned audiences. % 



- - 'Page 24 was retyped at The Center for Vocational Education, 
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NEW EDUCATIONAL DIRECTIONS 

^^Bowker 6annon and Associates Education Pro]tets 



BOX 307 



dRAWFORDSVlLLEJNDIANA 47933 



(317) 362-8877 



HovembeT 1, 1973 



. sNew Idu^tlonal plree t lens . (NED) is Intersstad In leaminfi of the-' 
Vf forts iaing e«ried out in the yarlous states caneernlng stmte^wlde 
Waljjatlons, voeationml-^teehnlcal pifO||rMfts at both t^he seeondary and 
post'^sedondary levels. ^ * ' 

^. " ■ - . ' . ■ 

Is your state eurrently liivalved In or has Ite^^letisd a eofl^rahensive 
gtate-^lde evaluatlra of vpeatlonal^tsehnleal progrMi^ In the state? 
if no^ la such an evaluation beings considered for lapleiBentatlon in the , 
foreseeable futu^? ttKymm^ Will you share wlth^NED descriptive" Inform 
mktioni interim or final reports ^ as well as data-g%thering teghnl^ues 
snd instruments for such a eos^rphe^nslve 'study? 

I look Jvward to hear tog from you ioan« , NED 5?ill report t© you'a 
suma^ of tjie ra^lfc. ^af wi receive ^f row this letter of inquiry. I 
fh«ik\you in^adyafic^ for your 'cofaperatlon. 

Sincerely yours ^ 




Judith M, Tho^aon 
Director of /Admlnla^ration 
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NEW EDUCATIONAL DIRECTIONS 

wBowker Gannon and Associates Education Projects 



BOX -307 



CRAWFORDSVILLEJNDIANA 47933 



(J17) 362^8877 



Odtobtfr- 10, 1974 



New EducatloMl Directions Is eurMntly working with tha Indiana 
State Boaltd of Vocational and Technleal Education* Our prassnt 
task Is to review literature conceraed with evaiuatlon of voefttlonal- 
teahnicai aduaatlon for grades -K-12 (deluding SK^mplary eareer 
education projects at the elem^tia^ level) , Ultimately the state will 
be developing a corapreheneivp. evaluation ^system. We are writing to 
,the various states requesting^ Infortiation they might have available* 

We are just beginning sb any Infonaation or faaterlals you ate able ^ 
to supply will be de#ply apprehlated. At this stage, we would like 
to know about the systpm(s)^you have developed (If you have one)^ 
Inf draat ion * concerning Tiow you went about developing it, how ths 
system works j time sehadules ,^purcM utilized, and qost sstioates ^ 
for developing and applying th6 ^stem- . '/ 

Thank -you in advance for your assistance totf^I laok^ forward to hearing 
from you soon* \ ' : 

' Sincerely J . 



SKRiaa 



landra K. Rorti 
MmffiiBtrative Associate 
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NEW EDUCATIONAL DIRECTIONS 

^^Bowker Sanno n and Assochtes Education ProjBCts 

BOX 307 CRAWFORDSVILLE, INDIANA 47933 (317) 362^8877 



December 6, 1974 



New Educational Directions hai written to the various atates 
coneerning m evaluation gyatam for vocatlonal-tecltiaieal edueafion, 
Mr* Phil Mmn^ Coordinator of Evaluation, Indiana Departmant of 
Public Instruction, has suggested that we write to. you dlrectly^^out 
the enclosed request for InforMtlon' as we have not yet received a 
response from 

Thank you in advance for your assistance , and 1 look forward to 
hear tog from you soon* . s 

Sincerely* 



Sandra K._Roth 
Administrative Adsoclate 



sm:aa 
Enclosure 
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REQUEST FOR ASSISTANCE 



New Educatibiial Dlractlons Is currently working with the Indiana State 
Board of Vocational and Technical Education. Our preMnt taak is to 
review llteratura concerned with evaluation of vocational -^technical 
education for grades K-12 (including exemplary career education projects 
at the elementary level), Ultlinacely tha state will be developing a 
compi^ihenslve evaluation system. We are Siting to the' various states 
requesting Information they might have available. 

We^aie just beginning so any Information or materials you are able 
tof supply will be deeply appreciated. At this staga^^ w^j^uld like 
to. know about the systems you have developfd, W one^l^ available ^ 
Information concerning how you went about developing it* how the ^ 
system works ^ time schedules, sources utllli^a, and cost estimates for 
daveloplng and applying the system will be ^test helpful to us. 
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56. 

Distributive Education Evaluation Form j South Carolina State Department of Educa- 
tion ^ Columbia* ' - 

57. 

Dyer, Marilyn L. , The Concepts of Evaluation as Related to the Total Curriculimi , 

Individual Courseg ^ and" Students ^ presented at "Keeping on Course* Report of 
the 1970 Regional Worfi.^hops of the Council of Diploma Ptograini/* National 
League for Nursing, New York. New York, 1971. (ED 050 27^) 

.58. 

Edsall. Richard H. , A Guide for Local Progr^ Evaluation, Information Series 

No. 73^ Ohio State Undversitys Center for Vocational and Technical Educatlonj 
Columbus^ sponsoring agency Nation?il InatitAate of Education (DHEW), 
Washington, D.C, , 1973. (l© 078 130) 

Education for Employgient in Ohio . Ohio Advisory Council for Vocational Education 
ThI rd Annual Report 197g-1973V Ohio State Advisory Council for Vocational 
^ Edlication" Columbus, sponsoring agency " National Advisory CouncM on Voca- 
tional Education, WashinGton, D.C.; Office of Education (DlffiW)^ Washington,! 
D*C,, November, 197£*'. (ED 080 659) 

60* . ^ ^ 

Eorployer Survey ^ State , of Flcrida, Department of Education, Division of VocatnrineLl, 
Technical, and Adult Education, Tallahassee* 

61. 

Enrolljiient Contract Checklist , Accreditation Cwmii-ission, National Association of 
Trade atid Technical Schools^ August l6, 1971- 

62. ' 
Enrollment Form , Office, of the State Superintendent, Vocational and Occupational 

' Skills rTB^leneC, Montana. 

Evaiuation ^ Objective Approach , National^ Lea^e for Nursing, ^New York, New 
York, 1972. (ED 06l U30) 

6U, 

An Evaluation ^ Lincoln Area Vdc^tioTial School , Pike County. Information anj 
ft^Qcedures ^ Missouri State Department of Education. . 

65. " " 

Evaluation of Exeiiqplajy Fronects , State of Iowa, Departo^nt of Public In^truf^tT nn, 
" Des Moines 5 Janu^^y, 19T2. 
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Xn' Fhralu ation of Post-'SeenhAfir>- Vocati onal— Technical Educatien in H^^tco, 

December,' 1969. (ED 037 5^3) 
!nii Evaluation Ptoccss. Missotiri State Department of Education, Division' of Yoea- 
gg tional Educat-ior , 

The T-i-aauation Report of the S_guth Carolina Advlaoiy Council on ycTeati_o_nca jiiv3 
■ Technical Edueatipn ,' Souttf Carolina Advisory Council nn Vocational and lv.,jhnl- 

■ cai Education, ColuHajia, Novemberj 1973. 
Ev aluatlpn Service Center for Occupational Education . Fin_y, Report, Massachusetts 
"~ University, Center for Occupational Education, Amherst, sponsoring agency — 
Massachusetts State Department of Education, Researah Coordinating Unit for 
Occupational Education, Boston; New York State Education Departoent, New York 
Research Coordinating Unit, Albany, June 30, 197E. (ED 085 5^1) 

Evaluative Questionnaire for Agricultural Education , Missouri State Department of 
Education,. Division of Vocation^ Education. 

Evkuatlve Questionnaire for Business and Office Education . Missouri State Depart- 
"mei^ of Education, Division, of Vocational Education. 
--Svaluati^ Q uestionnaire for Combination Cooperative P>ccupatlonal Education, 
"C:- MissSuri State DepeurtmeDt of Education, Division of Vocational Education. 
^^Iuatlv4 QuestlonnM-re for Distrubutiva Education , Missouri State Department of 

E due it ion. Division of Vocational Education. 
Evaluative auestlonnalre for GuldMge. Missouri State Department of Education, 

Dlvisldn of Vocational Education, 
EvIluatlYe (^estlonnaire for Wealthy Oecupat Ions Education , Missouri State Depart- 

ment o? Education, Division of Vocational Education. 
Ev^ative euestibnnaire for Economics , Missouri State Department of Educa- 

tion, Division of Vocational Educ at ipn, ■ = 

Evaluative Qilestlonnaire for Vocational Education - Administrablon, Missouri State 

^artonent Qf Education, Division of Vocational Education. ^ . 

Evkuative Questionnaire for Vocational Education - Curriculum, Missouri State^ 

Depai^ment of Education, Division of Vocational Education. 
Evaluativ e Questionnaire for Vocational Education -- Fhilosophy, Missouri State 
qq EipStment of Edueatlon, Division of\yocatlonal Education. ^ 

Evaluative Queetionnalre- for Vbcatlonal IS^^.M^^^idlt^' 
~ Ml^Suri State De p^ r^n^ of Education, Division of Vocational Education. 

Fikatra, Clarence, The UCLA ^co^^ School Bisi Demoos^trat i on ^^e^ 

for an IntroductiSrr trAlligd wkth Care^K California University Division 
orv^ational EduStTSn, Los Angeles, sponsori^ asency -- Calif ornia_fat ate 
Department of Education, Bureau of Industrial EU^eation,Saeramento| Office 
of Education (DIffiW), Division of Comprehensive and Vocational Education Re- 

Qg search, Washington, D.C. September, 1973. 4S»-©B2 ^016) 

Forbes Rov Phase IIIj nr^wt-affeetiveness EvquaMdn , p^sented in "A Guide to 
° l^aSakonT^aJiacl^iis Inf oration ^^^^V^^^ 1°°^°^- 



Frvaluationi Massachusetts inioKna^ion reeuuo,wftwjf« v-.- ' V o 
First Technical Proeress Report,* edited by Kath^yn H ^d-Eueene R. 
Spless, Massachusetts Research Coordlr^ating Unit for Vocational Education, 
Boston, September, 1969. (™ 03^ 071) 
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r^c^^---:... \ iagram of ^ Vgcational Education Delivery ^stss* F^kman 
"^"'■'"7-.' -: J^and Hills, CallTornia/ OctoBif/lTj T^TT^ ^ 

General Evaluatiytf Quest i orinalrn for Vocati onal Education ^ Missouri State Depart^ 
ment of Education/Division of Vocational r'luaation.- 

85- ^ ^ - ' 

Gillie J Angelo C. ; Ed, ^ The AnnugLL Pennsylvania Conference on Poat-becond ary 

Occumtlonal Educatiori (gnd) , PennsyTvania State University, Center for the * 
Study of Hi"gher Education, "University Park^ June, 1971- (ED 056 291) 

66. ' - ^ 
Griessmans B. Eugene , Planned Chani?e .1 n Lov/- Income Rural Areas : An Evaluati on ot 

Concerted Services " in Training and Education . ' Center Research Monograph No. $5 
North Carolina" State UnTvftrsltyV Center for Occupational Educ at ion 5 Raleigh , 
sponsorin^^ afency Office of Fducation (DffilV/)} Washington^;-D/C* , 1969* 
(ED OUr^ 906) 

67. . / 

Grieve, William , Measurement and Appraisal of Competenei , presented at the 

NationaJ Senr nar on Resa^arch" fn EvaluatTon of Occupational Education^ North 
"Carolina State University, Center for Occupational Education^ Raleigh, span- 

sorjng agency Office of Education (DHEW), BuresAi of Reeearch, Washington,^ 

D*C./l968, "(ED 060 178) , ^ 
SS, _ . _ 

Guide for Public Schools In Planning Programs of Occupationaa Education for In-School 

^Students ^ Texas Education Agency^ Austin, June^ 197?* 

69. ' " _ ^ 

A Guide for Reviewing a District Vocational . Education System, California State ^ 

Department of Education, ^acrajnentOj 1972- 

90. . 

Guidelines for Conducting Periodic Follow-Up Studies in the VT^ System, 1970. 

. (ED OU7. 093) ^ . 
91. 

- Guidelines for Local EvaluataQn , Virginia State Board of Education^ Richmond, ^ 
September 12, 1973^ 

9?* 

Guidelines for Preparing an Evaluation Plan for ft-ojects. Supported Under Partg C 
~ and P. L, 90^576 ^ "New Jersey State Department of Education. 

93. ^""^ ^ ^ ^ -- - . 

Guide to ^inproving Vocational Education Evaluation , Arkansas University, College 
~ 'o~Educatiori, Fayette vi lie-, sponsoring "agency — Office of Education (DIffiW), 
Bureau of Research, Washington, D*C*, December, 1970* (ED OU? 153) 

Gutcher, G. Daae and' Wlllii,^ George E., A Survey of Post-Secohd^y Vocational- 

Technical Teachers in Texas ^ Texas A and M University^ College of Education, 
College Statlon.,"^anuary, 1973* (ED OSp 533l i ^ 

Hamlin, K. , Citizen Evaluation of Public Occupational Education ... Center 

Monograph No> 1, North Carolina' State Uni vers i%. North aarpllM State Univer- 
sity Center for Occupational Education, Raleigh, sponsoring agen^ Office 
of Edueation (DHEW), Division of Adult. and Vdcatlonal Research, Wishlngton, 
D,C., 1967* (ED 033 2SB) 

96 

Handbook for Evaluating Disadvantaged and Handicapped Students fw Planning Occu- 
— ^patlonil EducatlQli Programag North" Carolina State Board of Education, Depart- 
' ment of PUbllc Instruction,, Raleigh, February, 1970. (ED 05S 3^?) 

97 

Hanftbook for E-i/aluating Instructaon , Los togeles City Schools, California, -1971* 
. (ED 0^ 556} ^ 
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9b: - . . ^ 

Hajad'bQOk for Self *EvaJ.uat i on of ^TOgrami^and Ser^c^s ^ Disady aataffed thm 
HydicaOT g d at CoBOTunli ^ ^orie ges iimg^^ ^endinenti" ' "* 

19bO t t^fe^Q I^aw 90Vg76 J> TadlQck AaeociateSj Inc*; Los Altoa, CallforiS'a^ 
iponioriog'^l^ency — California Coordinating Unit for Occupational leaearch 
and Dayelopmentj SacraiQento" Foothill ComnMnaty College District^ Cupartino, 
California, April 15, 1972- (ED 073 SUl) ^ . 

99. , ' ^ ^ ^ ■ 

Hawkridgej David G, and others , A Study of Selected Rrogrami for Vocational Educa - 
tion in Secondary Schools , Fin^ Re^rt y toerican Institute for Reseij^ch . 
, in Behavioral Scienees, Palo Alto ^ Californias si^nsoring agency'--- Office of 
Education (DKEW)^ Bureau of Elementary and Secondary Education, Washington, 
D,C,, Jamia^, 1970. OUl 155) ^ 

100. 

Heinkelj Otto A, , Priority Detamlnation for Vocational Education Through a Fqt^^^ 
Needs Aasesament Process^ Final PrOelect Report ^ San D lego Com^nity College 
District, Ceaifornia Reaew'ch Office, San DiegOs 1973* (ED 086 295) 

101, 

Hopkins, Charles 0* , Ed* D. , Relating Accountability to the Ob^^ectivefi of Vooational 
Education ^ Oklahoma State Department of Vocational and Technical Educations 
December 7, 1971. ^ 

lOK* . 

Hu* Tsh-Wei -and others, A CQst^Effectivenegs Study of Vocational Education , Final 
Rep ort , Pennaylvania State Univereity, Institute for ReaegLrch on Human Re* 
ao'^. ^^B, University Park, sponsoring agency Office of Education (DHEW), 
Washington, D*C*, October, I968, {ED 029 093) V 

103* / ' ... - ' . . 

^rplement at 1 on New Dealga^ For The Challenge Of The 1970s ( Trade and Iftdustrial 

Education ) ^ American Vocational Association, Washington, D,C., 1558* \^ 

(ED 0S8 282) 

lOU, ^ ^ . _ ^ \ 

Indira State Advliory Council for Vocational Technical Education ^ Third Aimual 

Report s Indiana State AdvlBory Council for Vocatiqnal/Technleal Education^ 

Indianapdlis * 

10?. ^ _ : . , " 

Ina ■ ana State Plan for Vocational Education , Fiscal Year 1975 . Indiana State^ BoM*d 

qL >7bcational and Technical Educationi Indiahapolli, 197^^' 
106. ^ - ^ 

InfOTO ation ^d Policies , Western Aasoclation "of Schools and Colleges, Accrediting 

"Coiomfssion for Secondary Schools, Burlingaae^ California- 
107- ' ^ 

Instru^aantg and ftrooedures for the Evaluation of Vocational/Technical Institutions 
and ^ograjiis , American Vocational Association, Washington, D.G., iponsoring 
agency National Center for Educational Research «id Davelopjient (_DIffiW/OE), 
Washington, D,C,, 1971, (ED 08I BUo) 

108,..^ ' ^ ^ 

Int&rlm Evaluat ion Guide for Institutional Assessment , North Ceivtrai Asflociation of. 
\ CoUtges and Secohdai^ Schools, Comrission on Institutio of Higher Educa- 
tion, Chicago, Illinois, April, 197^- 

109. / ■ 
Introduction to Vocations Program Evaluation 197g-1973 s Evaluation Report No. 86,' ^ 

Hawaii Department of Iduoatloh, Office' of Instructional Services/Evaluation 

Section, ^Honolulu, November, 1973. 

110. - ' ■ / ' 
Ittner J I^ed E* , ft-oject to Develop a Cogt Benefit Model for Vocational faogrias at 

College of i^^eda . FlnaX Report » Mameda College, California- sponaoring 

agency OfiHei ot Education (Dffiw), Division of Conprehenalve and Vocational 

Research, Waahinpton, D.C*| PeraJ,ta Junior College District, OiW^wids Cali« 

. fornia, .taaguet, 1972. (m ajB jn^Y ' - - ^ , 
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rrKut- -■ ■■ ■ arid Fortuiif* , Jim Phase ^i ^vnirr%m Eva luati on . pi^^oent.ea m 

.. :> Evraluatlon- Massacl:usotts In forma tTon'Tt'diback Syatem for Voea- 
-ation. First T^chmcal FroereiS Report,'' edited by Kathryn H. and, 
Eii.^^ene Bm^zz, M^issnnhua^tts Besparch CoordJnatin^ Unit for Vocatlot.al 
Education, Boston, S.^ptiymb.-r, l^nti, (ED 03^ 071 ) ^ , , 

11?. 

Jordan, Clintan . Accountability or Leadership ? Proceedings of the Annual ;:.tato 
CQnferenca on 'Educational Sesearch "Accountability and th# Currlculiim" (23rd), 
California Advisory Council on Educational Research , ^Parlingamej sponsoring 
agency California Teachers Association, Burlineame, November, 1971^ 
(ED oSa 3S2) ' . , 

Jorgenson, Elaine and others , M Evaluation ^ Oklahoma' a Exemplary Vocational 
Education and Qceupatlotf ^ " Origntation ^ograjn . Tulsa Public Schools 
1970^1973 , "Oklahoma" State University ^ Fall, 1973. 

Kauftnan ■ Jacob J. , C c s t ^E f f a c 1 1 ve ne s s Analysis as a Method for the Evaluation of 
Vocational and Technical Education , sponBoririg acency Office of Educp.t.ion 
(Dliw), WasMHgton, D.C, /December 10^ I9681 (ED 029 983) 

115* / . ^ : 

Klit, Jdhn A. and V/entling, Tim L, , The Development and mpleme ntatlon of a State- 

wlfle Ev.aluation Syatem , 
116* /" ' "" 

Klit, John A,, Wentling^ Tim L. , and Langsjoen, Marcia A., Impact of a State-^ 

D irected Evduation System ^ presented at the American Educational Research 
AkuoQi ation~Convention, Chicago, April 17, 197^* ^ ^ 

117- / ' 3 - ' 

K005 Wo Y6n, Fh* D., Controversies in Educational Evaluation , presented at 

f Leadership and Change in Vocational-Technical Education: Administrators' 
J/orkshoii for Building Research and Evaluntion Into Local Practice (Edison, 
kew Jersey, May 5-6, 1970). Final Report," Rut.^ers, The State Univer- 

(sity. New Jersey Department of VocatS onal-Techni cal Education^ New Brunnwick, 
Isponaorinr:- agency New Jersey State Department of Eduoation, Division of 
/Vocational Educat:ion, Trenton, July, 1970. (ED 0U8 U61) 
lie J 

j . A Manual for Self-EvaJuatlon of Career Development ^o.iect , 

I ~ ~ *^ " ■ " ■ " ' - ----- - — 

Kot%, Arnold, Oecu-patlonal Education — Planning and ProBraitming . Volume II, 
Stanford Research Institute, Menlo Park, California, September, 1967 . 
' (ED 017 73'+) ■ 

Kraft,, Richard H. P. , Perspectives on ft-ogress : Caree r Education in Flurida; 
Statewide Evaluation of Vocatlonal-TBChnieal Education 1970-1971 , Florida 
Stat^ tJniveraity, Department of . Educational Administration, Tallahassee, 
sponsoring agency — Florida State Board of Health, Jacft'sonville, 1971. 
(ED 056 235) = . - ■ ' . 

121. 



Krehs,- A. H. , Model for Evaluation of Secondary Sc hool Programs of Voeational 

EducatloiTir ^Tiguiture , Maryland Unversity, Agricultural Experiment Station 
, College P e?k, October, I969. (ED 05° 2?^) 

Lewis, James P. and Dittenhafer, Clarence A., Evaluation of Selected Career M- 

source Centers In Pennsylvania , Pennsylvania Departoent of Education, Buredu 
of Vocational E^catlon, Eesearoh CoordlnatinB Unit, Harrlsburg, December,- , 
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Little, J. K'^im^th and I'/hinfifeld, Richai^d W,, Fol inw-^Hp of 196^ Graduates of 

Wisconsin aaii^^xe ^ vocational, Tachnical^ Adult Educ ation , Wisconsin 
.University, Center for Studies iff' Vocaticmkl "and^ Technical Educati or. L^adison 
f^^^ June, 1970, (ED 0U7 111) • ' " " ' 

LoQ^, Long-Range Plans, Wisconsin State Drpartmont of Public Instruction, Madiaon. 

Eo^ Range Flanningr Guiae Plan .cf Service for Vocational Education , Alaska Depart- 



ment of Education 5 Juneau, 



Haley ^, Donald, ^ Impleinentation , and Further Develo-OTent of Experimental Cluster 
Concept Programs Thrnugh Testing and E valuat ion "inc ludi nr PI ac emint ^anf"^^ ^ 
FQllow-.lTp af Sub,1eDt£i . Cluster Concept froject . ftiase IV, Finan&port , 

Mar^aand University, College Park, sponsoring igency — Off3ee of Education 

197. (^^^''0* ,Sureau of J^esearch, Washington, D.C^ Au^st, 196a* (ED Ok2 Okl) 

Malay j Donald and Mietus, Walter Q., ^ Implement at Ion and Further Deyeio^ent of 
Exparlmental Cluster ^ Concept Programs Through Actual Field Testing^ and Eval~ 
uation; at the Secondary School Lavei ; Cluster Concept ^oJect .~ HiIi? ^ 

^ III. Interim Report , aponsoring agency Office of Educ ati on ( DfffiW ), - 

128. C-p,, August^ I96C. (ED OPS 963) 

A Manual- To Be tJaed ,;ln the Evaluation ^ Thirty^ f -^^^^ Comprehensive High Schdols in 
Wisconsin Vhicb Participated in a Three~Ye - ) Progrro at High School ~ 
Vocational Educatjon , VMsconsin ^tate Depi^ ' - = ^C^f Public Instruction, 
Madison, February^ I968* - (ED ^^S 659) 

Maryland Advisory Council on Vocational-Technical. Education TOird Annual Evaluation 
Ragort , .lQ7g> Maryland State Advisory Council on Vocational-Technicai Educa-""; 
tion, BCLtimora, sponsoring agency Bureau at Adult, Vocational^ and Technl- 
cal Education (DHEW/OE), WashinFton, D.C. , 1972, (ED 076 8OI) 

■ The Maryland Vocatia^al Education Research Coordinating Unit , Fiscal Year 1971 
t July 1, 1970- June 30, 197?T." Final Rr'port, "Maryland Vncat 1 nnal^FducatTon 
- Research Coordinating' Unit ^ Baltimore,^ sponsoring agency Maryland State 
_ Department of Education, Baltimore; ofece of Education (DHEW) , Wash int^ ton 
2^,^ 1971. (ED 066 585) ^ . ; ^. ' . 

A Master 'Plan Cor the Development of Vocatlpnal-Techn i cal Education in Mexico , 
oterlinf: Institute, Washington, D,C, , sponaorinf/ agency — EconomlG Develop- . 
13^_ ^^P^ Adjnlni.stration (DOC), Washington, D,C,, September, I969, (ED 079 550) 

McKinney, Floyd and Mainnebach, Alfred J,, Using Student Conmi ttees in Educational 
^ = ^ Prograjti Evgquatlon , March 1, 1973, (ED otib 79h) 

McKlnney, Floyd and Oglesby, Charlies, Developing and Conducting Follow-^Up StudieB 
of Formc^r Students , Kentucky Research Coordinating Unit," Lexington, September' 
1971. 056 2hg) ^ ; " ^ ^ 

Meyer, Judy, S elf -E valuat i on Guide for Local Districts for Vocational Education of 
Handl dapped Students , Houston University, Center "for Human Resources, Houston, 
Texas, sponsoring agency — Texas Education Agency, Division of Occupational' 
Research and Development, Austlni 1979* (ED; 07U 275) 

Miller, Melvin Dfe and Budke, Wesley Eugene, Job. Placement and Followup of Secondary 
Sohoal Vocational Education Students . PREP- 32 » National Center for Educational 
GoiMunlcation (DEEW/OE), Washlngtonj D.C., sponsoring agency Office of 
Eaucation (DHM), Washington, D.C,^ July, 1970, {m O62 581) 

Minnesota Vocational-Technical S e 1 f^E valuat Ion Procedures , Minnesota. 

k¥^k ±n -72. Missouri Advliory Councir '^^.Vocatlonal Education Third Annual -^ 
\ , Report , December , 197g\ Miaaouri State Advisory CouncH on Vocational Bduca- 
• tioQ, Jefferion City^ aponsoring agency --Office of Education (DfffiW), 
Washington^ p*C, , December ^ 1972. (ED O76 792) 



138, ^ ^ ' ' ' ^- ■/ / . ^ ' ^ ' r ' . ... 

L97I Annual E^/,aluat-loji Repo rt cm Vocational Educatj-on in Georgia ^ Georgia State 
5F ~.7 Advisorv^ Coiincil on Voca.tional Education, Atlantaj sponsorlfng agtney Office 
H}- ' of Education ( CHEW )/ Washington, D,C^/l971. (ED O65 68I) ' . 

Sorth Dakota Annual Report * Stdte Board for Vocational Education, Bismarck, Fiscal 
ikO, . . . ' 

gorth Dakota Vocational Education Asgeggment Sy stein . North Daicota State Board for 
• " Vocational Education,. Bianmrck. 

Lf+1. 

riie Northwest Association of Secondary and Higher Schools , Comlsalon on Secondary 
~ " Schools ^ Handbodk , 3 tandards~f or Accreditation , Constitution ^ The Nor;bhwest 
..Association of Becond^^ and Higher Schools, Secondary Commission, Pendleton, 
i Oregon, 1971* . . — 

Lte, . - ^ - ^ 

Norton i :Daniel P, and Watley, Donivan J* , The Efficiency and Efficacy of the Eval ^ 
uation Practices of the 1111 noi s Division of Vocational and Technlcgl Educa - 
tion " Final Report , Educatidnal" Testing Service^ Evanston, Illinois,, apon- 
soring agency lUinoii State Advisory Council on ^Vocational Ed-ytcation, 
: Springfield/ December 21> 1972-i (ED 079 ^29) 
IU3. 

Dccupatlonal Education . . ft'Ograms ; toage of the World of Work , Volume II , Develop - 
ment of Instrfainents and Evalurfbi on , Colorado State University, Ft. Colllns|^ 
: Rocky Mountain Educationai Lab7 Inc., Greeley , Colorado, sponsoring agency 
- Office of Education (DfffiV/), Bureau of Research, Washington, Au^st, 
1969. (ED p3U 885) 

Jccupational Skills > Program Evaluation 197£^73* Evaluation Report No, 85, Hawaii 

Pepartment~~of Education, Office of Instructional Services/Evaluation Section, . ^ 
Honolulu^ November, 1973* . ^ 

1^5. . ^ /. : \ ' 

Jceupatlonal Education Prosrani Criteria . . ^ Assessment Instalment , North Carolina 
f \l1tate De^ Public ^nstruct ion, pivision of Occi^ational Education, 

fe- Raleigh, 6ctdber,-il97U. ; . 
IU6. ^ - 

Df'fice Ocoupatl£?ris Evajuation , South Carolina State Department of Education^ 
■V Columbia. ■ ^ ' - ^ 

3hlo I Progras Re^rjew for toprovement' s D e vel_o:^ent , and Expansion in Vocational 

~ Education ^~OMo State "gepartoent of Effucationj Division of Vocational Education, 

^ .^ Columbus [1970, (ED 085 W^) 

ma. . ^ ^ _ ^ ■ ■ \ . 

)tiid ProCT^ Review for Improvement. ^ Development and Expansion in Vocational - 
X, ; Educatipn* ^ Broeedure Gufde for/V.. E, D. Coordinator , Ohio State Department - 



of 'Education, 'Dl?^sion of Vocational. Education , Cdltunbus, March, 1973* 

3hio faogram ' Review for !toprovement ^.. D^elo'^eht ^ and Expansion in Vocational 
P^v Educatlon and (Mldance >^ J^ogram ' Review ^r » Ohio State Department 

. of "EducatiorT, DivisTon of Vodational .Education, Colutobua = ^ 

156. \. ^ ' . ^ ^ . ^ ^ ^ ■ ^ ;^ - ' - ^ - . _ 

)rilb . Rrograjn Review for J^provtoent s JfeyelO'^ent , ap^l ^yp arislon in Vocational 

Education , 7 Rosram Review for ' TSabitlet Making and Millwork , Ohio State 
KiS-v Dapartment o£^ Educatloh, Columbui* 

I^BS^ogram Review for Snprovement , Development , and Expansion .in Vocational 
^a lEaucatlon . ' Program ^ Review for Clerical Services, Ohio State Department of 
0mMeit±on^ Divi:sl,bn of Vocational Education/ Columbuaw:. . . ^ 



, 152. - 

. Q^^o l^ogyain Ravi mi for . Improvement^ Devclo pmant ^. and E xpan sAon in V ocat ional 

Education , Pr ogr airPHg vi e w for -Cooperatiye OfficeT;5ucatrQn^COE )g Ohio Stato 

Department of Education, Diviaion of Vocational Education. Columbus. 
15i* . - 

Ohio Progr^ He^ew for ^prowmiint ^ Development ^ and Eypansion in Vncational= 

Education . Program Reviev for Medifcal Assi sting » Ohlo~St ate Department of 

^ _i Education* Division of Vocational Education, Coluinbuo* 

O hio ^Qgram Review for toprovement ^ Development^ and Expansion in Vocational 

Education . Rrogra m Review for Office Duplicating , Ohio State Department of 
Education, Division of Vocational Education ^ Columbus, 

Ohio Program Review for ^ Improvement , Development » and Expansaon in Vocational 
Education . RrogrMi Review for Radio and f7V, ,~ Ohio State Department of 
Education^ Division of Voc at! onaZl Educations Columbus* 

Ohio Program Reviaw for ^provemerit^ Development ; and Expansion in Vocational 

■ Education , firogtam Review for Sma ll " Engine Repair , Ohio State Depajftment 
, c*? Educations Division of Vocational Education, Cqlumbus, 

Ohio Program Review for Improvemeni , Development ^ and Expijislon in Vocational 

Eiducation * Proga^am Review for ypcatlon^ Education Admi n i s t r a 1 1 o n , Ohio State 
'^■g Department of Edu^t^on^. Division of Vocational Education," Columbua* 

Oliver, J, Dale and otherSj Voc at 1 o nal Edu^at ion E vaj^uat Ion Pi'o J o c 1 1 Annual 
Repor t gi^cal Year 1973 a Virginia Polytechnic Institute and State 
Unlversl-ty , .Blacksburgj sponsoring agency Virginia State Department of 
Education, Division of Vocational Education^ Rlchmondj Septmber, 1973- 

— ^ (ED O6U 360) _ 

Organization and Operation of a Local ^ogrm of Vo.eatlonal Education ^ New York 

State University^ Western Hew York School Study Council, Buffalo, sponsoring 
160 " Office of Education (DHEW), Washington, D.C, y I968. (ED 0?2 06I) 

Orientation for Review Py tic^ijanta , California State Department of Education, 
jg^^ Vdcatiohal Eduafttlon~7ectiorrr 11 ' 

iEAPOL ( l^ogram Evaluation ^ the Performance Objective Level ) User ^ s Manual s ^ 
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